He says it 's his last fight . And it 's been his hardest .

After 43 years in Congress , New York Rep. Charlie Rangel said the campaign for his 23rd term is his final one .

The 84-year-old `` Lion of Harlem '' wants two more years in Washington , but on Tuesday , voters will decide if his time is up .

He 's a founding member of the Congressional Black Caucus , and the former chair of the powerful Ways and Means Committee .

But the legend of Harlem politics is on shaky ground .

Rangel running for 23rd term in the House

Rangel says he 's ready this time

He 's been bruised before . In the 2012 Democratic primary , victory came by a thin margin . Rangel beat state Sen. Adriano Espaillat by fewer than 1,100 votes .

The congressman said he did n't put up much of a fight .

`` I did n't have a campaign last time . When he told me he was running I was in the hospital in Columbia Presbyterian with a viral infection in my spine . ''

This time , Rangel said he 's ready .

`` Well , I do n't have a walker . I do n't have a spinal injury . ''

Rangel has also had more time to recover from a humiliating censure in 2010 from the U.S. House of Representatives following a series of ethics violations .

2014 midterms : What 's at stake

Gearing up for Round 2

Since 2012 , Espaillat has been gearing up for Round 2 . This time , the underdog said he 's counting on an upset and making comparisons to one of the biggest ones ever in boxing :

`` Sonny Liston was a big , bad bear when he got into the ring , but -LRB- Muhammad -RRB- Ali was faster , smarter than him , '' Espaillat said . `` He was able to draw circles around him , and he shook up the world and he changed boxing . ''

If voters want a change now , it may be a measure of how much the district has changed since Rangel went to Washington in 1971 .

Harlem 's rich African-American history now seems distant

Neal Schumacher knows the streets of Harlem as well as anyone . He gives walking tours , and he often points out the fact that the neighborhood is , '' ... not the Harlem that I grew up in , but a Harlem I embrace . ''

A falling crime rate has accelerated real estate development in the neighborhood , but affordable housing remains among the most pressing issues for many with roots in the community .

`` People are concerned about gentrification , about rising housing costs , about some of the mom and pop shops closing . About being able to afford to live in the community where you are now , '' said Schumacher .

Harlem has long been known for its rich African-American history . But to Harlem historian Jacob Morris , that past feels more distant today .

`` In the 20th century Harlem was incontrovertibly ... the cultural capital of Black America . ... Is it still ? I would say now it 's the custodian of that great history . ''

Both say racial and ethnic politics not part of race

Today , the demographics are different . So is the district itself .

Part of the Bronx was added to Harlem 's district before the 2012 primary . The 13th Congressional District now has a Hispanic majority .

If elected , Espaillat would become the country 's first Dominican-American congressman .

Both candidates insist that racial and ethnic politics should n't determine the outcome of the race .

`` We never had a political battle in my congressional district in 43 years based on where you were born or what religion you have , '' Rangel said while campaigning Saturday .

2014 midterms : Complete coverage

Turnout could be key

According to a NY1/Siena College poll released the week before the primary , the vast majority of voters said the race or ethnicity of the candidates makes no difference to them ,

The same poll shows Rangel has a 70-point lead with blacks and a 5-point lead with whites , while Espaillat has a 24-point lead with Latinos . Overall , the poll gives Rangel a 13-point lead -LRB- 47 % -34 % -RRB- against Espaillat .

Insiders suggest the race is tighter than that and too tough to predict .

A key factor : turnout is expected to be low . Both campaigns will have to battle to get voters to the polls on Tuesday . The congressional race is the only item on the ballot .

@highlight

Rep. Charlie Rangel is running for 23rd term in the House

@highlight

Challenger is same one who came within 1,100 votes of unseating Rangel in 2012

@highlight

Challenger state Sen. Adriano Espaillat predicts an upset on Tuesday

@highlight

Rangel was first elected to seat in 1971 but district has changed recently